Current digital three-dimensional modeling software, including gesture based interfaces, predictive and suggestive interfaces, recognition systems, image understanding, constraint based systems, and curve creation techniques, enable the creation and manipulation of extremely sophisticated geometric objects, and has been widely adopted by practitioners in industry. Such software programs include: Pro/ENGINEER™, SolidWorks™, and AutoCAD™. However, as illustrated below, there are disadvantages in using such prior art technologies for the purpose of three-dimensional sketching for effective and easy integration into paper/pencil based sketching processes involved in many creative processes.
The user interfaces for these three-dimensional modelers, however, are typically quite complex, relying on the standard WIMP (Windows, Icons, Menus and Pointing devices) interaction metaphor to perform all tasks, typically using a mouse and keyboard as the primary input devices.
The principal disadvantage in using the current digital three-dimensional modeling software is that the ability of individuals involved in creative processes such as designers (in many different fields of endeavour), artists, and architects (collectively referred to in this disclosure as “designers”) to work in rich physical media is generally not fully leveraged in current WIMP based solutions, as particularized below.
Given that the fidelity of current digital tools often does not approach that of physical tools, many traditional methods continue to be used in the industry. In particular, in the early stages of designing a three-dimensional model, paper and pencil is typically used to quickly create and iterate on concept sketches. However, there is currently no mechanism for easily integrating the concept sketches into the three-dimensional modeling pipeline and related digital processes. The disadvantage is that at present, pencil and paper sketches are generally not used for anything more than an external visual reference when designs are created by or for designers using digital creative tools. For example, in the case of architects, their pencil and paper sketches are often used by support staff to create three-dimensional models built painstakingly from low level graphical primitives. This not only represents additional work, but can result in the loss of certain layers of or richness of data.
An improved sketching interface is discussed in “Teddy: A Sketching Interface for 3D Freeform Design”, Takeo Igarashi et al., 1999, ACM SIGGRAPH, p. 409-416 (“TEDDY”). Systems like TEDDY enable the creation of three-dimensional models using a gesture and sketching interface that allow users to quickly and directly sketch their models and offer useful suggestions for subsequent actions based on the current context. However, even such interfaces do not adequately leverage the invaluable pencil and paper sketches.
Gestural interfaces depart from the WIMP metaphor by inferring actions from user input strokes, rather than relying on explicit manipulation of onscreen widgets. Generally, gesture-based interfaces, used in two-dimensional pen-based applications recognize specific stroke shapes as gestures and replace them with predefined primitives or invoke editing operations such as undo. For example, the system described in “SKETCH: An interface for sketching 3D scenes”, R. C. Zeleznik et al., 1996, ACM SIGGRAPH 1996 Conference on Computer Graphics and Interactive Techniques, p. 163-170 (“SKETCH”) provides a gesture based interface for creating three dimensional scenes consisting of stacked geometric primitives.
The TEDDY system for sketching freeform three-dimensional models improves upon the usability of SKETCH by using a smaller but more easily understood gesture set, at the expense of being able to create relatively simple objects. The essential idea in TEDDY is to use the freeform strokes as an expressive design tool. The user draws two-dimensional freeform strokes interactively on a blank canvas specifying the silhouette of an object and the system automatically constructs a three-dimensional polygonal surface model based on the strokes. TEDDY also allows suggestions to be inferred based on hints provided by the user at the immediate past timestep.
SKETCH and TEDDY are similar in that users start from a blank canvas and explicitly create every single piece of geometry. However, in TEDDY, ideas were extended from the predictive interfaces of the SKETCH system by providing suggestions of geometry that could be used next. Both TEDDY and SKETCH systems operate on the principle that the suggestions can be inferred, provided that the user's hints correspond to a matching algorithm.
The main disadvantage inherent in the TEDDY and SKETCH systems is that they tend to reproduce the feature curves of the input image on a blank canvas with the designers drawing all parts of the new model. A further disadvantage of blank canvas and matching algorithm based solutions is that the algorithms in these systems are not robust enough to recognize the hints from human-drawn sketches so as to infer suggestions from the user's hints.
Another prior art system is generally known as “Pegasus” and is described in “Interactive beautification: a technique for rapid geometric design”, T. Igarashi et al., 1997, ACM UIST Symposium on User Interface Software and Technology, p. 105-114, and “Pegasus: A drawing system for rapid geometric design”, T. Igarashi et al., 1998, Extended Abstracts of ACM CHI Conference on Human Factors in Computing Systems, p. 24-25 (collectively “PEGASUS”). Three-dimensional computer renderings of sketches using the PEGASUS system generally improve on the hand-drawn strokes by interactively extending the notion of beautification and prediction. PEGASUS beautifies hand-drawn strokes by inferring desirable geometric properties and predicts the next operation based on the surrounding context. It also generates multiple candidates to facilitate these processes. However, the disadvantage of generating too many potential candidates on the scene is that it clutters the scene and presents problems for the average user.
There are also recognition-based systems for handwriting, speech, and text input techniques that also face the problem of resolving ambiguity of among multiple candidates. Multiple candidates are also generated when searching for good possibilities within large parameter spaces in other graphic applications.
Another disadvantage of the prior art inventions is that even if they are able to generate multiple candidate suggestions, the user interface is not conducive to integration of such multiple candidate suggestions into the creative process of designers. For example, in each of the TEDDY and PEGASUS prior art systems, the multiple candidate suggestions are produced in a separate location on the screen thus forcing the user to context switch between locales; or the multiple candidate suggestions are presented as representative examples of a large continuous space as in the recognition-based systems described in “Design galleries: A general approach to setting parameters for computer graphics and animation”, Marks, J. et al., 1997, ACM SIGGRAPH Conference on Computer Graphics and Interactive Techniques, p. 389-400.
Also, the current digital tools generally require that users learn a new skill set, which constitutes a relatively significant barrier to adoption and use.
There is therefore a need for method, system and computer program that permits designers to use their pen on paper concept sketches meaningfully in the construction of computer implemented models, both two- and three-dimensional. There is a need for such a method, system and computer program that dynamically provides intuitive suggestions based on inference of subsequent geometric possibilities. There is also a need for an interface that enables dynamic interaction with paper and pencil sketches in a modeling pipeline. There is a further need for a method, system and computer program as stated before, that includes a graphic user interface that is easy to use.